It is common in today's service management applications to require data loads. Examples of data that needs to be loaded include records of new persons, new assets, security groups, etc. Typically, these loads are manually performed, but the manual process leads to a number of problems, some of which are as follows.
With complex data models, some data loads depend on the success of one or more prior data loads, further complicating the responsibility on the data loader to ensure previous loads were successful.
For referential integrity purposes, some data loads depend on data already existing in the data source, driving additional responsibility on the data loader to confirm that the requisite data is already present in the data source before loading.
Currently automated data loaders attempt to address some of the issues encountered with manual data loading, such as automatically performing validation checks and auto-scheduling the loads, etc. However, currently automated data loaders do not provide capabilities that would enable data loads to be performed at a higher level of functionality.
RAJU et al. (U.S. Publication 2008/0256575) disclose a system that includes a data loader and a data slicer. The data loader is configured to receive and process raw program guide data, and to store the processed program guide data in a data store. The processing includes generating a unique program identifier for each program represented in the raw program guide data. The data slicer is configured to generate a program guide data configuration from the processed program guide data. The program guide data configuration includes program guide data organized into a plurality of data structures based on categories of the program guide data.
KIM et al. (U.S. Pat. No. 7,069,179) disclose a data extracting and processing module that extracts necessary data from a database and generates an analysis table. The data extracting and processing module has a process analysis table generating module and an activity analysis table generating module to extract data and generate the analysis table. The preprocessing module searches characteristics of data on the basis of the data extracted by the analysis table, removes unnecessary attributes, divides instances if necessary, and converts a digital variable into a symbolic variable by dividing sections.
CARLEY et al. (U.S. Pat. No. 6,701,345) discloses downloading data from the a user station to a server. It is determined whether another load process is being concurrently executed by another user station. If it is determined that a load process is being concurrently executed, a notification is sent to the user station. A notification is also sent to the user station that initiated the concurrently executing load process. At least one of the load processes is suspended upon detecting the concurrently executed load process. At least one of the load processes may be allowed to continue upon receiving a command to continue from the user station associated with the suspended load process. A data management template corresponding to files/records is selected. The data management template may include a listing of all records/files that should be loaded. Alternatively, the data management template may specify particular content of the files/records that must be matched for verification. As an option, the data management template may specify specific particular sizes of the files/records. It is validated that all of the records/files to be loaded match the data management template. The records/files are sent to a database for loading in the database upon validation that the records match the data management template.